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Abstract  -  This  paper  takes  stock  on  long-term  research 
activities  targeting  the  development  of  support  systems 
for  the  intelligence  analysts  and  the  decision  makers 
involved  in  situation  analysis  and  sensemaking ,  It  first 
reviews  two  notional  models  of  situation  analysis  and 
sensemaking ,  which  provides  insights  into  these 
multifaceted  processes ,  It  then  discusses  the  natural  fit 
of  knowledge-based  systems  technologies  with 
intelligence  analysis ,  and  describes  some  previous  work 
in  knowledge  representation ,  automated  reasoning ,  and 
multiple  hypothesis  situation  analysis ,  The  current 
research  efforts  towards  the  integration  of  automated 
reasoning  components  within  the  multiple  hypothesis 
framework  are  described.  Finally ,  work  in  progress  for 
the  development  of  a  multiple  hypothesis  sensemaking 
support  system  on  a  service-oriented  architecture 
platform  is  presented. 
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1  Introduction 

During  the  last  decade  or  so,  research  activities  have 
been  conducted  at  Defence  R&D  Canada  -  Valcartier 
(DRDC  Valcartier)  towards  the  development  of  support 
systems  for  the  intelligence  analysts  and  the  decision 
makers  involved  in  situation  analysis  and  sensemaking. 
These  activities  cover  a  broad  spectrum  going  from 
modeling  the  situation  analysis  process  to  generate 
insights  about  all  facets  of  this  complex  task,  to  the 
specification  of  a  knowledge-based  framework 
appropriate  for  sensemaking,  through  to  the  development 
of  a  service-oriented  platform  dedicated  to  the 
implementation  of  the  most  promising  concepts  in  these 
areas.  This  paper  reviews  the  main  concepts  of  situation 
analysis  and  sensemaking,  discusses  key  elements  that 
have  been  previously  investigated,  and  describes  current 
R&D  work  targeting  the  development  of  a  multiple 
hypothesis  sensemaking  support  system  (MHS3). 

The  paper  is  organized  as  follows.  Section  2  provides 
definitions  of  situation  analysis  and  sensemaking  and 
briefly  describes  two  notional  models  that  can  be  used  to 
better  understand  intelligence  analysis.  Then,  as 
knowledge  management  and  exploitation  are  key  aspects 
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of  situation  analysis  and  sensemaking,  Section  3  quickly 
reviews  some  relevant  work  by  the  authors  in  knowledge 
representation  and  automated  reasoning.  Previous  work  in 
multiple  hypothesis  situation  analysis  is  also  summarized 
in  Section  4,  and  the  current  research  efforts  regarding  the 
integration  of  the  automated  reasoning  components  with 
the  multiple  hypothesis  framework  are  described.  Finally, 
Section  5  discusses  the  implementation  of  the  MHS3  on  a 
service-oriented  architecture  platform,  and  Section  6 
provides  some  concluding  remarks. 

2  Situation  analysis  and  sensemaking 

Situation  analysis  (SA)  is  defined  as  a  process,  the 
examination  of  a  situation,  its  elements,  and  their 
relations,  to  provide  and  maintain  a  product,  i.e.,  a  state  of 
situation  awareness  (SAW)  [1-2].  The  SA  process 
encapsulates  that  part  of  the  intelligence  cycle  (or  the 
decision-making  cycle  if  one  is  more  involved  with  this 
aspect)  that  is  concerned  with  understanding  the  world. 
There  is  a  real  situation  in  the  environment,  and  the  SA 
process  will  create  and  maintain  a  representation  of  it,  the 
situation  model.  In  turn,  a  situation  is  defined  as  a  specific 
combination  of  circumstances,  i.e.,  conditions,  facts,  or 
states  of  affairs,  at  a  certain  moment. 

At  the  highest  level,  making  a  strong  parallel  with  the 
work  of  Endsley  regarding  situation  awareness  [3],  the  SA 
process  can  be  decomposed  into  four  subprocesses: 
situation  perception,  comprehension,  projection  and 
monitoring  (note  that  we  are  talking  processes  here,  not 
states)  [1-2].  Situation  perception  has  to  do  with  the 
“acquisition”  of  the  situation  through  data/information 
collection  with  various  sensors  and  other  sources. 
Situation  comprehension  is  about  further  developing  one's 
knowledge  of  the  situation  with  respect  to  both  its  nature, 
i.e.,  the  inherent  character  or  basic  constitution  of  the 
situation,  and  its  significance  or  meaning,  i.e.,  the 
importance,  the  consequence,  the  impact  and/or  the 
implication  of  the  situation.  This  subprocess  must  be  able 
to  grasp  the  nature  of  the  situation,  and  to  derive 
operationally  relevant  meaning  and  significance  from  the 
results  of  situation  perception.  Situation  projection  must 
produce  an  estimate  of  future  possibilities  for  situation 
elements,  based  on  current  trends.  Finally,  situation 
monitoring  has  to  do  with  watching,  observing,  or 
checking  the  evolution  of  the  situation  in  order  to 
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ultimately  keep  track  of,  regulate  or  control  the  operation 
of  the  SA  process. 

Figure  1  presents  a  notional  model  of  the  SA  process  [1- 
2].  From  a  data-driven  perspective,  SA  entails  integrating 
and  interpreting  the  whole  spectrum  of  source  data  and 
information,  ranging  from  radar  returns  to  political 
factors.  The  SA  process  thus  encompasses  a  vast  range  of 
activities,  from  the  detailed  signal  processing  associated 
with  object/entity  acquisition  and  tracking,  to  intelligence 
interpretation.  Simply  put,  the  process  must  provide 
answers  to  a  great  number  of  questions:  What?  Who? 
How  many?  How  big?  Where?  What  structure?  When? 
What  is  it  doing?  Why?  Build  up?  What  could  it  do?  How 
soon?  What  is  outstanding?  What  has  changed?  Delta 
from  expectations?  What  is  going  wrong?  and  many 
others. 


Figure  1:  Notional  model  of  situation  analysis  [1-2] 

As  suggested  with  Fig.  1,  the  SA  process  consists  of 
numerous  dependent  and  independent  subprocesses,  or 
capabilities,  at  multiple  levels  of  abstraction.  Every 
subprocess  can  itself  be  further  decomposed  hierarchically 
into  multiple  subprocesses.  These  SA  capabilities  must  be 
integrated  and  interleaved  into  an  overall  processing  flow. 
From  a  data-driven  perspective,  Fig.  1  suggests  a 
sequential  process  (with  a  clockwise-rotation  “feel”) 
starting  with  acquisition/collection,  and  then  going 
sequentially  through  structured  description  (integration/ 
abstraction),  classification/recognition,  assessment, 
projection,  impact,  and  monitoring.  But  it  doesn't  have  to 
be  like  that.  Actually,  one  should  note  that  there  are  no 
arrows  on  the  diagram  of  Fig.  1.  In  a  sense,  the  SA 
process  has  no  beginning  and  no  end  per  se.  The  SA 
process  may  start  anywhere;  there  are  multiple 
asynchronous  entry  points.  Moreover,  it  is  not  necessarily 
a  cycle.  Actually,  human  information  processing  in 
operating  complex  systems  is  seen  as  alternating  between 
data-driven  (bottom-up)  and  goal-driven  (top-down) 
processing.  This  alternation  is  viewed  as  critical  in  the 
formation  of  SAW.  In  goal-driven  processing,  attention  is 
directed  across  the  environment  in  accordance  with  active 
goals  [4].  The  decision  maker  actively  seeks  information 


needed  for  goal  attainment  and  the  goals  simultaneously 
act  as  a  filter  in  interpreting  the  information  that  is 
perceived.  In  data-driven  (or  stimulus-driven,  reactive) 
processing,  perceived  environmental  cues  are  processed  to 
form  SAW  and  may  indicate  new  goals  that  need  to  be 
active.  Dynamic  switching  between  these  two  processing 
modes  is  one  of  the  most  important  mechanisms 
underlying  SAW. 

Making  a  parallel  with  the  service-oriented  architecture 
(SOA)  software  engineering  paradigm  (cf.  subsection 
5.1),  the  SA  capabilities  (or  subprocesses)  of  Fig.  1  could 
be  regarded  as  “services”  having  some  fair  degree  of 
autonomy.  In  that  sense,  there  is  a  requirement  that  the  S A 
capabilities  must,  at  a  minimum,  communicate  with  one 
another  or,  ideally,  cooperate  with  one  another.  The 
capabilities  at  the  higher  levels  of  abstraction  must  build 
on  the  results  produced  by  the  lower  levels,  and  they  also 
feed  conclusions  back  to  these  lower  levels  in  order  to  fill 
in  unknowns. 

A  detailed  description  of  the  SA  model  shown  in  Fig.  1 
can  be  found  in  [1-2].  Given  its  crucial  importance  within 
the  SA  process,  threat  analysis  has  been  further 
investigated  and  discussed  [5-7]. 


2.1  Sensemaking 

Sensemaking  is  the  ability  or  attempt  to  make  sense  of 
an  ambiguous  situation  [8].  More  exactly,  sensemaking  is 
the  process  of  creating  situational  awareness  and 
understanding  in  situations  of  high  complexity  or 
uncertainty  in  order  to  make  decisions.  Based  on  results 
from  a  cognitive  task  analysis  and  verbal  (think  aloud) 
protocols,  Pirolli  and  Card  [9]  provide  a  description  of 
intelligence  analysis  as  an  example  of  sensemaking. 
Figure  2  represents  their  notional  understanding  of  the 
analyst’s  process.  This  is  a  broad  brush  characterization  of 
the  whole  process  they  have  seen  across  several  all-source 
analysts  as  a  way  of  approximating  the  process  and 
orienting  their  studies. 
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Figure  2:  Notional  model  of  intelligence  analysis  [9] 

In  the  model  of  Fig.  2,  the  rectangular  boxes  represent 
an  approximate  data  flow.  The  circles  represent  the 
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process  flow.  The  processes  and  data  are  arranged  by 
degree  of  effort  and  degree  of  information  structure.  This 
is  a  process  with  lots  of  back  loops  and  it  seems  to  have 
one  set  of  activities  that  cycle  around  finding  information 
and  another  that  cycles  around  making  sense  of  the 
information,  with  plenty  of  interaction  between  these. 
This  process  diagram  summarizes  how  it  is  that  an  analyst 
comes  up  with  novel  information. 

The  data  flow  shows  the  transformation  of  information 
as  it  flows  from  raw  information  to  reportable  results  [9]. 
External  data  sources  are  the  raw  evidence,  largely  text  by 
the  time  it  reaches  the  all  sources  analyst.  The  “shoebox” 
is  the  much  smaller  subset  of  that  external  data  that  is 
relevant  for  processing.  The  evidence  file  refers  to 
snippets  extracted  from  items  in  the  shoebox.  Schemas  are 
the  re-representation  or  organized  marshalling  of  the 
information  so  that  it  can  be  used  more  easily  to  draw 
conclusions.  Hypotheses  are  the  tentative  representation 
of  those  conclusions  with  supporting  arguments. 
Ultimately  there  is  a  presentation  or  other  work  product. 
Basically  the  data  flow  represents  the  transducing  of 
information  from  its  raw  state  into  a  form  on  which 
expertise  can  be  applied,  and  then  out  to  another  form 
better  suited  for  communication. 

As  previously  mentioned,  the  overall  process  is 
organized  into  two  major  loops  of  activities  [9]:  (1)  a 
foraging  loop  that  involves  processes  aimed  at  seeking 
information,  searching  and  filtering  it,  and  reading  and 
extracting  information,  possibly  into  some  schema,  and 
(2)  a  sensemaking  loop  that  involves  iterative 
development  of  a  mental  model  (a  conceptualization) 
from  the  schema  that  best  fits  the  evidence.  Information 
processing  in  Fig.  2  can  be  driven  by  bottom-up  processes 
(from  data  to  theory)  or  top-down  (from  theory  to  data). 
The  analysis  of  Pirolli  and  Card  suggested  that  top-down 
and  bottom-up  processes  are  invoked  in  an  opportunistic 
mix. 

2.2  Models  overlap  and  complementarity 

Clearly,  the  notional  models  of  Figs.  1  and  2  overlap  on 
many  aspects.  But  there  is  also  a  fertile  complementarity 
between  them.  While  the  situation  analysis  model  of  Fig. 
1  focuses  more  on  the  components  of  a  situation,  the 
model  of  Pirolli  and  Card  gives  emphasis  to  the  cognitive 
processes  of  an  intelligence  analyst  that  is  trying  to 
acquire  and  develop  knowledge  about  such  components. 
Both  models  could  thus  be  used  together  to  structure  the 
intelligence  analysis  process  and  provide  an  adequate 
foundation  for  the  development  of  situation  analysis  and 
sensemaking  support  systems. 

Actually,  these  models  can  be  used  to  better  understand 
the  intelligence  analyst  problem  space,  i.e.,  to  map  the 
problems  (e.g.,  user  requirements,  horizon  scanning,  etc.) 
on  the  models,  and  also  regarding  the  technology  space, 
i.e.,  to  map  the  technology  (e.g.,  COTS,  GOTS,  industry, 
academia,  varying  technology  readiness  levels)  on  the 
models.  This  way,  problems  are  mapped  to  potential  tools, 
and  tools  are  mapped  to  the  capability  they  offer.  This 


enables  the  generic  application  of  tools  to  various  areas  in 
the  problem  space  and  helps  to  distil  out  where 
technology  already  exists  to  address  particular  problems. 
It  highlights  where  there  are  problems  but  nothing  in  the 
technology  space  addressing  it  (leading  to  further 
development),  and  the  potential  technologies  that  could  be 
of  benefit  when  advanced  in  technology  readiness  level 
(TRF).  The  tools/technology  space  can  then  be  readily 
mapped  to  some  service-oriented  framework  (cf. 
subsection  5.1)  so  that  a  structure  for  the  service  delivery 
can  be  articulated. 


3  Knowledge-based  framework 


The  idea  of  awareness  ultimately  has  to  do  with  having 
knowledge  of  something.  In  addition  to  the  cognition 
facet,  awareness  is  also  linked  with  the  notions  of 
perception  and  understanding/comprehension.  While 
perception  is  defined  as  the  result  of  becoming  aware  of 
something  through  the  senses,  comprehension  and 
understanding  are  both  defined  as  the  knowledge  gained 
by  grasping,  with  the  intellect,  the  nature,  significance,  or 
meaning  of  something. 

Such  considerations  have  led  us  to  develop  and  adopt  a 
knowledge-centric  view  of  situation  analysis  and 
sensemaking.  Within  our  research  activities,  aspects  of 
knowledge,  and  how  it  can  be  acquired  from  experts, 
formally  represented  and  stored  in  knowledge  bases  to  be 
exploited  by  computer  programs,  and  validated  have  been 
initially  discussed  in  [10].  The  benefits  and  issues  related 
to  the  use  of  knowledge-based  systems  technologies  in  the 
context  of  situation  analysis  support  systems  (SASSs) 
were  then  discussed  at  length  in  [11-12].  The  key  related 
aspects  of  knowledge  representation  concepts,  paradigms 
and  techniques,  and  reasoning  processes,  methods  and 
systems,  and  knowledge  and  ontological  engineering 
techniques  for  use  in  developing  knowledge-based  SASSs 
were  further  investigated  and  discussed  in  [13-15]. 


a  Create  1 
V-  Generate  \  J 


Figure  3:  Knowledge  management  and  exploitation 

Instead  of  providing  a  crisp  definition  of  knowledge 
management  and  exploitation  (a  difficult  task  that  often 
leads  to  controversy),  Fig.  3  illustrates  what  it  is  that  one 
typically  does  with  knowledge.  As  for  the  models  of  Figs. 
1  and  2,  Fig.  3  can  be  used  to  provide  an  adequate 
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foundation  for  the  development  of  situation  analysis  and 
sensemaking  support  systems,  and  to  guide  the 
development  of  specific  knowledge  management  and 
exploitation  (KME)  tools,  which  can  eventually  be 
mapped  to  some  service-oriented  framework  (cf. 
subsection  5.1). 

3.1  Knowledge  representation 


as  metadata  to  provide  a  wide  variety  of  contextual 
information  regarding  uncertainty,  temporal  and  spatial 
issues,  security,  the  source  of  the  fact,  etc. 

Among  the  contextual  attributes  attached  to  a  fact  are 
those  related  to  the  temporal  domain  (some  of  which  are 
illustrated  on  Fig.  6).  Such  attributes  are  used  to  support 
the  management  of  the  fact  over  time,  and  temporal 
reasoning. 


The  object  of  knowledge  representation  (KR)  is  to 
express  knowledge  in  computer-tractable  form,  such  that 
it  can  be  exploited  [13].  Figure  4  illustrates  different 
categories  of  knowledge  that  are  relevant  to  support  the 
situation  analysis  activities  of  an  analyst  in  the 
intelligence  domain. 
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Figure  4:  Categories  of  situation  analysis  knowledge 


Figure  6:  Fact  attributes  to  establish  the  temporal  context 


To  enable  the  representation  of  such  knowledge,  the 
knowledge-based  framework  developed  at  DRDC 
Valcartier  allows  for  the  use  and  manipulation  of  a 
number  of  knowledge  representation  building  blocks  [16]: 
ontologies,  facts,  inference  rules,  text-based  templates, 
etc.  Among  these,  the  notion  of  a  «  fact  »  (and  that  of  an 
«  atom  definition  »)  has  been  adopted  as  a  key  knowledge 
representation  paradigm  [16].  In  our  framework,  as  shown 
in  Fig.  5,  a  fact  is  defined  by  a  name  (i.e.,  a  label)  and  a 
list  of  arguments  with  a  precise  type  and  order. 
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Truth  Model  Continuity 
Discrete  Truth  Value 
Continuous  Truth  Value 
Uncertainty  Status 
Uncertainty  Type 
Uncertainty  Measure  Type 
Uncertainty  Measure  Value 
Conflict  Status 
Existence  Status 
Coupling 

Fogical  Dependency 

Fogical  Dependency  Requirement 

Utility 

-  Creation  Position 

Deletion  Position  Comment 

-  Reception  Position 


Figure  5:  Example  of  fact  attributes 

An  example  of  a  fact  could  be  « Is  Exposed  To  (Mike, 
Radiation,  Uranium)  »  to  formally  express  the  idea  that 
“Mike  is  exposed  to  radiations  caused  by  uranium”.  A 
number  of  attributes  (Fig.  5)  are  also  attached  to  the  fact 


Figure  7  illustrates  similar  contextual  attributes  that  are 
used  to  support  the  spatial  management  of  a  fact,  and 
spatial  reasoning. 


Figure  7:  Fact  attributes  to  establish  the  spatial  context 

3.2  Automated  reasoning 

Not  all  of  the  situation  elements  of  interest  are  directly 
observable  with  sensors.  This  is  especially  true  of  highly 
abstract  types  of  situation  elements  (e.g.,  enemy  intent), 
and  also  of  the  relationships  between  the  situation 
elements.  Those  aspects  of  interest  that  cannot  be  directly 
observed  must  be  inferred,  i.e.,  derived  as  a  conclusion 
from  facts  or  premises,  or  by  reasoning  from  evidence.  In 
this  regard,  Fig.  8  shows  the  environment  that  has  been 
created  to  enable  the  inference  of  situational  facts  through 
automated  reasoning  (ISFAR)  through  the  synergistic 
exploitation  of  the  complementary  strengths  and 
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expressiveness  characteristics  of  different  knowledge 
representation  approaches,  and  corresponding  reasoning 
paradigms  [17]. 
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Figure  8:  Inference  of  situational  facts  through  automated 
reasoning  (ISFAR)  [17] 

In  our  framework,  the  ISFAR  concept  has  been 
implemented  as  a  set  of  independent  services  (cf. 
subsection  5.1)  performing  rule-based  [18],  description 
logic  [19]  and  case-based  [20]  reasoning.  A  module  for 
kinematics  and  geospatial  analysis  has  also  been 
implemented  as  specific,  dedicated  inference  code  [17]. 

4  Multiple  hypothesis  sensemaking 

Uncertainty  makes  the  analysis  of  even  simple 
situations  difficult.  It  forces  intelligence  analysts  to 
formulate  and  manage  hypotheses  during  the  construction 
of  explicit  representations  of  real  world  situations  (Note 
that  the  generation  and  management  of  hypotheses  is  an 
explicit  component  of  the  sensemaking  model  proposed 
by  Pirolli  and  Card  [9]).  This  may  quickly  become 
overwhelming.  To  provide  better  support  to  the 
intelligence  staff,  the  main  concepts  behind  multiple 
hypothesis  tracking  have  been  revisited  to  develop  a 
proof-of-concept  prototype  of  a  multiple  hypothesis 
situation  analysis  (MHSA)  support  system  [21-23]. 

For  the  MHSA  prototype,  a  graphical  language  has  been 
defined  and  made  available  to  the  analyst  for  the 
construction  of  explicit  representations  (or  models)  of  the 
situations  under  examination.  This  language,  shown  in 
Fig.  9,  allows  the  intelligence  analysts  to  define  and 
manipulate  situation  model  components  (SMCs)  to  create 


graphical  representations  of  situations.  The  language  is 
limited  to  five  types  of  SMCs  that  can  be  used  by  the 
analysts:  1)  Element ,  2)  Undirected  Relation ,  3)  Directed 
Relation ,  4)  Relation  Origin  Connecting  Point ,  and  5) 
Relation  Destination  Connecting  Point.  Everything  that  an 
analyst  has  to  say  about  a  given  situation  must  be 
expressed  using  only  these  five  types  of  SMCs. 


Figure  9:  Graphical  language  for  situation  modeling  [22] 


Within  the  proposed  MHSA  framework,  one  says  that 
there  is  uncertainty  when  there  are  more  than  one 
possibility  for  a  given  situation  model  component  [21-23]. 
Moreover,  when  there  are  multiple  possibilities  for  a 
situation  model  component,  these  possibilities  must  be 
mutually  exclusive ;  if  one  is  true,  then  all  the  others  are 
necessarily  false.  When  there  is  only  one  possibility  for  a 
situation  model  component,  then  this  possibility  is 
considered  as  «  certain  ». 

Figure  10  shows  a  conceptual  view  of  the  proof-of- 
concept  prototype  of  a  multiple  hypothesis  situation 
analysis  support  system  (MHSA-SS)  that  has  been 
developed  at  DRDC  Valcartier.  Details  can  be  found  in 
[21-23]. 


Figure  10:  Conceptual  view  of  the  MHSA-SS  prototype 

Current  research  efforts  at  DRDC  Valcartier  investigate 
the  integration  of  the  automated  reasoning  components 
(i.e.,  the  ISFAR  framework)  with  the  multiple  hypothesis 
framework  (i.e.,  the  MHSA-SS  prototype)  to  develop  a 
multiple  hypothesis  sensemaking  support  system  (MHS3). 
Figure  11  shows  the  high-level  interactions  of  the  main 
components  required  to  achieve  MHS3. 
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Figure  11:  Components  high-level  interactions  for  MHS3 


The  MHS3  provides  tools  to  support  the  construction  of 
a  situation  model  representing  the  analyst’s  mental  model 
of  the  situation  (1).  Actually,  through  the  MHSA 
framework,  the  analyst  can  construct  and  maintain 
multiple  possible  situation  models  in  parallel  (2).  If  the 
formalism  of  facts  and  ontologies  (cf.  subsection  3.1)  is 
used  for  the  construction  of  the  situation  models  (3),  then 
the  automated  reasoning  components  (4)  can  be  applied  to 
distinct  sets  of  facts  corresponding  to  distinct  situation 
models  (i.e.,  distinct  hypotheses).  The  resulting  sets  of 
inferred  facts  obtained  from  the  different  reasoners  can 
then  be  used  to  generate  multiple  augmented  possible 
situation  models  (5)  supporting  the  maintenance  or 
augmentation  of  the  mental  model  of  the  analyst  (6). 


Multiple  Possible 
Situation  Models 

(Multiple  Hypotheses) 


V 


^  ...  ^ 


Figure  12:  Fact  containers  for  the  situation  models 

Figure  12  illustrates  how  the  distinct  fact  containers 
corresponding  to  the  distinct  situation  models  (or, 
equivalently,  the  distinct  hypotheses)  are  created  from  the 
different  knowledge  categories.  These  fact  containers  are 
required  for  step  (4)  of  Fig.  11.  Note  that  the  notions  of 
«  certain  »  versus  «  uncertain  »  facts  on  Fig.  12  refer  to 
the  MHSA  framework  where  «  certain  »  components  are 


contained  in  all  hypotheses  and  «  uncertain  »  components 
are  specific  to  particular  hypotheses. 

Each  individual  fact  container  of  Fig.  12  is  sent  to  the 
automated  reasoning  component,  i.e.,  the  multi-reasoner 
inference  (MRI)  component  of  Fig  11,  at  step  (4)  of  Fig. 
1 1 .  Depending  on  the  particular  facts  involved  and  on  the 
«  know-how  »  provided  to  this  reasoning  component,  new 
facts  will  potentially  be  generated  (the  inferred  facts  on 
Fig.  11).  Then,  at  step  (5),  the  initial  set  of  situational 
facts,  the  new  inferred  facts,  and  a  subset  of  the  reference 
facts  (the  ones  that  have  been  involved  in  the  generation 
of  the  inferred  facts)  will  be  used  by  the  situation  model 
builder  (SMB)  component  of  Fig.  11  to  create  an 
augmented  situation  model  corresponding  to  all  these 
facts.  This  is  illustrated  in  Fig.  13. 


Figure  13:  Generation  of  an  augmented  situation  model 

Facts  are  typically  statements  made  about  the  instances 
of  an  ontology.  If  all  of  the  facts  used  in  the  MHS3 
framework  are  forced  (by  the  system)  to  only  represent  1) 
a  situation  element  (i.e.,  an  atomic  unary  fact  with  only 
one  ontology  instance  in  its  argument  list,  e.g.,  «  Exist 
(Mokami)  »)  or  2)  a  binary  relation  (i.e.,  an  atomic  fact 
with  only  two  instances  in  its  argument  list,  e.g.,  «  Owner 
(Mokami,  John  Doe) »),  then  the  construction  of  a 
situation  model  from  a  set  of  facts  can  be  frilly  automated. 
Otherwise,  if  the  system  manipulates  arbitrary  formal 
facts  without  restrictions,  then  the  situation  model  builder 
component  will  require  human  intervention  to  deal  with 
the  potentially  complex  semantics  of  such  facts. 

5  MHS3  implementation  platform 

Many  of  the  concepts  discussed  above  for  MHS3  have 
been  initially  implemented  as  a  set  of  tools  within  the 
Multi-Intelligence  Tools  Suite  (MITS)  developed  by  the 
Intelligence  and  Information  (I&I)  Section  at  Defence 
R&D  Canada  -  Valcartier  (DRDC  Valcartier)  [16].  The 
MITS  is  a  federation  of  innovative,  composable  and 
interoperable  intelligence  related  tools,  which  are 
integrated  and  interleaved  into  an  overall,  continuous 
process  flow  relevant  to  the  intelligence  community. 

As  a  result  of  the  experience  gained  through  the 
incremental  development  of  the  MITS  so  far,  a  number  of 
lessons  learned  have  been  formulated  regarding  design 
and  implementation  approaches  and  enabling 
technologies.  Among  these,  the  need  for  a  more  suitable 
integration  platform  and  the  evolution  towards  a  different 
human-computer  interaction  layer  are  briefly  discussed 
next. 
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5.1  Intelligence  S&T  Integration  Platform 

The  efficient  development  of  a  suite  of  integrated  tools 
like  the  MITS  also  brings  forward  some  integration 
requirements  and  issues  at  the  software  system  level.  In 
response  to  these,  the  I&I  Section  at  DRDC  Valcartier  has 
proposed  and  developed  the  Intelligence  S&T  Integration 
Platform  (ISTIP).  Using  open  source  Web  services 
technologies  and  service-oriented  architecture  (SOA) 
design  principles,  the  ISTIP  provides  a  backbone, 
integration  reference  platform  for  the  iterative  and 
incremental  development  and  integration  of  the 
innovative,  loosely  coupled,  reusable,  composable  and 
interoperable  services  required  to  perform  tasks  in 
computer-based  intelligence  support  systems. 

The  SOA  paradigm  has  been  put  forward  for  the  ISTIP 
with  the  goal  to  achieve  a  number  of  (long-term)  benefits: 
improved  integration  (and  intrinsic  interoperability), 
inherent  reuse,  increased  leveraging  (including  leveraging 
of  the  legacy  investment),  streamlined  architectures  and 
solutions  (through  the  concept  of  composition), 
establishing  standardized  data  representation,  focused 
investment  on  communications  infrastructure,  “best-of- 
breed”  alternatives,  and  organizational  agility. 

The  services  developed  for  the  ISTIP  must  meet  a 
number  of  SOA  design  principles: 

•  Loose  Coupling  -  Services  maintain  a  relationship  that 
minimizes  dependencies  and  only  requires  that  they 
retain  an  awareness  of  each  other. 

•  Service  contract  -  Services  adhere  to  a  communications 
agreement,  as  defined  collectively  by  one  or  more 
service  descriptions  and  related  documents. 

•  Autonomy  -  Services  have  control  over  the  logic  they 
encapsulate. 

•  Abstraction  -  Beyond  what  is  described  in  the  service 
contract,  services  hide  logic  from  the  outside  world. 

•  Reusability  -  Logic  is  divided  into  services  with  the 
intention  of  promoting  reuse. 

•  Composability  -  Collections  of  services  can  be 
coordinated  and  assembled  to  form  composite  services. 

•  Statelessness  -  Services  minimize  retaining  information 
specific  to  an  activity. 

•  Discoverability  -  Services  are  designed  to  be  outwardly 
descriptive  so  that  they  can  be  found  and  accessed  via 
available  discovery  mechanisms. 

At  this  moment,  a  number  of  services  aligned  with  the 
principles  described  above  have  already  been 
implemented  and  are  available  on  the  ISTIP.  Many  more 
services  will  continuously  be  created  as  legacy  tools  and 
systems  are  converted,  and  as  the  research  activities  of  the 
I&I  Section  at  Valcartier  further  progress.  Ultimately, 
there  will  be  a  close  match  between  the  services  on  the 
ISTIP  and  the  notional  models  discussed  in  Section  2. 

5.2  Interaction  Interface  Layer 

Clearly,  providing  all  of  the  services  of  the  ISTIP  is  not 
sufficient  to  achieve  the  concept  of  the  MHS3.  One  also 
needs  some  human-computer  interaction  front-end  for  the 


exploitation  of  these  services.  This  front-end  is  called 
VOiiLA  (Visionary  Overarching  Interaction  Interface 
Layer  for  the  Analysis).  All  services  of  the  ISTIP 
combined  with  VOiiLA  constitute  the  MHS3  implantation 
platform. 

5.3  Implementing  MHS3  on  the  ISTIP 

Work  is  in  progress  to  implement  the  MHS3  framework 
as  a  set  of  services  on  the  ISTIP  (Fig.  14).  Many  of  these 
services  already  exist  on  the  ISTIP  at  this  moment. 
Examples  are  the  services  for  the  semantic  annotation  of 
unstructured  text  documents  (SATD),  the  automated  facts 
extraction  from  unstructured  text  documents  (AFEXTD), 
the  situational  spatial  features  management  (SSPFTM), 
the  situational  facts  and  ontologies  managers  (SFM  and 
SOM),  and  the  reasoners  of  the  automated  reasoning  suite. 
Other  services  are  currently  being  developed  to  support 
the  multiple  hypothesis  framework. 


Figure  14:  Work  in  progress  for  the  MHS3  on  the  ISTIP 

6  Conclusion 

This  paper  provided  definitions  of  situation  analysis  and 
sensemaking  and  briefly  described  two  notional  models 
that  can  be  used  to  better  understand  intelligence  analysis. 
Then,  as  knowledge  management  and  exploitation  are  key 
aspects  of  situation  analysis  and  sensemaking,  some 
relevant  work  by  the  authors  in  knowledge  representation 
and  automated  reasoning  were  briefly  reviewed.  Previous 
work  in  multiple  hypothesis  situation  analysis  was  also 
quickly  summarized,  and  the  current  research  efforts 
towards  the  integration  of  the  automated  reasoning 
components  with  the  multiple  hypothesis  framework  were 
described.  Finally,  work  in  progress  for  the 
implementation  of  the  MHS3  on  a  service-oriented 
architecture  platform  was  presented. 

Although  our  research  work  is  progressing  pretty  well  at 
the  moment,  the  automation  of  the  main  components  of 
the  proposed  MHS3  framework  remains  a  serious  research 
challenge  at  this  stage.  This  is  especially  true  regarding 
the  automated  generation  of  hypotheses  from  observation 
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data,  and  automated  generation  of  situation  models  from 
sets  of  situational  facts. 

Finally,  it  is  worth  mentioning  that  others  are  also 
proposing  similar  knowledge-based  approaches  to 
intelligence  analysis  and  sensemaking,  including  the 
development  of  service-oriented  solutions.  In  this  regard, 
the  work  of  Bray  [24-25]  is  of  particular  interest. 
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